The state of Federation
=======================

:author: Jessica Tallon
:date: 2015-08-03 11:40

It's been a `long time <http://mediagoblin.org/news/pump-api-progress.html>`_
since there has been any news on the state of
federation, so here's an update on where MediaGoblin's at and some
technical aspects of federation.  We've been working with the
`W3C Social Working Group <http://www.w3.org/Social/WG>`_ to define
the future of federation, and part of my work there has been to
work on the `ActivityPump <http://w3c-social.github.io/activitypump/>`_
standard.  There's more to say on that and why we're investing time
there, but this blogpost will mostly be about MediaGoblin and
federation from a technical perspective.

Over the last year I've been building up the foundations of
federation, creating the necessary APIs and infrastructure needed. I
am currently finishing the last of that, modifying the models to
handle federated data. This is a challenge as  the models were
designed without federation in mind. There is a lot of new data that
is being stored as well as remote and local versions of the models.

Once I've finished working on the models I'll be starting the work on
actually getting the media and comments federating across instances.

The core of the federation and APIs are "Activities", these are
created all the time right now in 0.8.0. Every time someone posts
media, comments, tags or updates anything. These serve two main
purposes:

- *Command language:* instructing the server to do something
- *Log:* telling clients and other servers what has been done

The activities at their core at subject predicate object, for example:

::

    Chris posts an image

The activity also usually contains the audience targeting data
(i.e. who it's for). These are represented in JSON with elements being
nested as other JSON objects, lets take a look:

.. code-block:: js

    {
        "id": "https://mediagoblin.com/api/activity/1"
        "verb": "post",
        "objectType": "activity",
        "actor": {
            "id": "acct:cwebber@mediagoblin.com",
            "displayName": "Christopher Allan Webber",
            "objectType": "person"
        },
        "object": {
            "id": "https://mediagoblin.com/api/image/1",
            "objectType": "image",
            "url": "https://mediagoblin.com/u/cwebber/m/mediagoblin-shirt-close-up-and-badge/",
            "fullImage": "https://mediagoblin.com/mgoblin_media/media_entries/814/IMG_1011_modified.jpg"
        },
        "to": [
            {
                "id": "https://mediagoblin.com/api/user/cwebber/followers",
                "objectType": "collection"
            }
        ]
    }

This might look overwhelming but it's the same basic subject predicate object
sentence above. Chris (the actor) is posting (the verb) an image (the object),
it also contains some extra information like IDs so they can be referenced and
the audience which is a a collection of people that follow Chris.

Once we have this activity, what actually happens, you ask? Each user has an
inbox and outbox like you do with email. Step by step walkthrough of what
would happen when someone posts an image from a desktop or mobile client:

1 Client to server
------------------

back a preliminary object to use in the API.

1. The client creates an activity that at minimum would look like
   this:

.. code-block:: js

    {
        "verb": "post",
        "objectType": "activity",
        "object": {
            "id": "https://mediagoblin.com/api/image/1",
            "objectType": "image",
        },
        "to": [
            {
                "id": "https://mediagoblin.com/api/user/cwebber/followers",
                "objectType": "collection"
            }
        ]
    }

2. This activity is then posted to the client's inbox.

3. The server receives the activity, creates an ID, and attaches the actor and
   other metadata, then saves it.

2 Federation!
-------------

where we're all on our own servers. This is what the server (once it's
finished) will do once it gets the client's request:

1. The server looks for whom it's sent to (the "to" property) to get a list of
   people that the object has to be sent to.

2. The activity is then sent to each persons inbox.

As one of Chris' followers you can now open up the website or any client and
see Chris' image. Commenting on the image will produce an activity which is
sent to Chris, who subsequently sends it as an update to everyone else.

Phew! we're done, that's how the API and federation works in MediaGoblin. We've
successfully posted an image from a client and had it sent around and hopefully
you're able to see how a comment or something else would work in this system
too.

This is just a high level overview, there are many technical problems
such as access control (who's able to view content), verification of
objects (checking what we've been sent is correct), etc. Federation
doesn't come easy but we think it's well worth it.
